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"^ . Abstract. In this paper we develop a theory to describe innovation processes in a 

network of interacting units. We introduce a stochastic picture that allows for the 

j^ , clarification of the role of fiuctuations for the survival of innovations in such a non- 

C^ ' linear system. We refer to the theory of complex networks and introduce the notion 

I ' of sensitive networks. Sensitive networks are networks in which the introduction or 

'■^ , the removal of a node/vertex dramatically changes the dynamic structure of the 

^ ' system. As an application we consider interaction networks of firms and technologies 

O I and describe technological innovation as a specific dynamic process. Random graph 

, ^, . theory, percolation, master equation formalism and the theory of birth and death 

processes are the mathematical instruments used in this paper. 

> 

lO ' 1 Introduction 

(N 

~^ ' This paper is devoted to the interdisciplinary theory of selforganization pro- 

i^». , cesses, paying particular attention to stochastic effects connected with inno- 

^^ ' vations in network systems. In our understanding "selforganization" is the 

f^ , spontaneous formation of structures 127128145) . An "innovation" , in a general 

system-theoretical understanding, is the appearance of a new species, of a 
2 . new mode of behavior, of a new technology, of a new product or of a new 

idea etc. |;i')ll7llslltJI;M |. 

Technological innovations are considered as the basic driving process for 
rH ' economic evolution and growth. In economics, innovation networks |5UI72j 

Q , and networks economies "86" have been discussed widely in the last decade. 

O ' However, a unified understanding of socio-economic networks is still missing 

| 1(J0I . "For many economists the study of networks is limited to the analysis 
of the functioning of physical networks such as the railway, the telephone 
system or the internet for example" 1691 . Although networks are thought to 



5h 

j^ ' be constituted by sets of actors and by links |1(J(J| , the very nature of these 

actors and the links between them varies among different authors. Nations, 

institutions, firms, products, or individuals may represent the actors. The 
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links can be defined quite differently. Innovation in such a context is mostly 
seen as an outcome of complex networks with heterogeneous actors. 

In this paper, we define innovation not as a product of a network activity 
but as a specific process in the formation of a complex network. Thus, in- 
novation is a process in the structure formation of a complex networks. We 
describe innovation as a process which dramatically and decisively changes 
the composition of a network. Accordingly, the network will behave differ- 
ently depending on which innovation has been introduced. More specifically, 
we propose a network description for dynamic processes in a system. By 
means of this system-theoretical perspective the appearance of innovations is 
related to certain mechanisms in the growth and change of networks. So, we 
create a network picture for dynamic processes as the emergence, survival or 
extinction of innovations which have been also considered in the mathemat- 
ical theory of biological evolution, in population biology and other fields of 
complexity theory. 

In this paper we will follow a special approach to describe innovations in 
evolutionary systems. We start in section 1 with an abstract definition of an 
innovation. We will then show that our approach is related to recent develop- 
ments in statistical physics which best can be described as an emerging field 
of complex networks theory |105| . Thus we will first give a survey of network 
approaches ranging from chemical networks |12()l4()j to biochemical webs ^Hj : 
protein webs 0| , food webs |23 1 to the structure of the internet 03] and the 
world wide web [3| up to general approaches from statistical mechanics [2]- 

In the second part of the paper we reflect upon structure and relations 
of socio-economic networks in a static (purely structural) picture. We will 
discuss some earlier results of random graph theory and percolation theory. 
We derive statements about connectivity and strongly connected components. 

In the third part of the paper, and in connection with our special inter- 
est in innovation processes, we will develop a dynamic network theory and 
in particular the theory of "sensitive" networks in more detail. The term 
"sensitive" networks denotes networks which are sensitive to the introduc- 
tion or removal of one or few nodes or edges, or in a more general context, 
to the occupation or leaving of a node. Specifically, sensitivity is linked to 
the question of whether a node (a species, a mode of behavior, an idea, a 
technology) is occupied by at least one individual or not. We will show that 
this problem is of relevance for the modeling of innovation processes. The 
dynamics of innovation processes may be described by stochastic equations, 
which can be formally treated within the framework of statistical physics. 
Some generalizations are possible. 



1.1 Innovation a System Theoretical Approach 

In economics, innovation is mainly understood as technological innovation 
describing the introduction of new technologies, products and production 
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processes. The differentiation between invention and innovation relates inno- 
vation to tlie economic exploitation of new ideas. However, it is also possible 
to look at innovations from a more general, evolutionary point of view. Ziman 
gives one example for such an approach when he writes "Go to a technological 
museum and look at the bicycles. Then go to a museum of archaeology and 
look at the prehistoric stone axes. Finally, go to a natural history museum 
and look at fossil horses. In each case, you will see a sequence, ordered in 
time of changing but somewhat similar objects." jlHlj . p. 3. However, not 
every change is an innovation. In this paper, we follow a system theoretical 
approach to innovation. In this framework, innovation is something new to 
the system and most essentially the emergence of an innovation changes the 
state of the system dramatically. In other words, the actual state of the system 
becomes unstable and a transition to a new state occurs. To define an inno- 
vation we first have to define the state of the system. Here, we again choose a 
very specific approach. We represent the state of the system as a point in the 
high dimensional occupation number space 27 35 . In this space a coordinate 
axis is attached to a certain type i of elements (with i — 1,2, ...,s, natural 
numbers). The occupation numbers are represented on this axis. 

To describe technological innovation we have to ask for a re-specification 
of this abstract concept. For socio-economic systems, the axes of the state 
space refer to different possible taxonomies. For instance, an axis i can rep- 
resent a certain technology from a set s, of different technologies present in 
the system. With such a technological taxonomy, competition processes be- 
tween technologies can be described |102I16| . The carriers of this competition 
process are firms using different technologies and competing with their prod- 
ucts on a market. This way, we link back to an economic understanding of 
an innovation process that "requires insight into system dynamics grounded 
in a variety of firm competencies and behaviour and a variety of demand" 
|103| . p. 5. Let us note that the state space concept can be applied to quite 
different processes. The type i might also stand for the size class that a cer- 
tain firm belongs to. Then, growth processes of firms are in the focus of the 
description. Moreover, the type i may represent a certain group in society. 
Formation of political opinions |129j or emergence of norms and violence in 
groups iHB! are then considered. Innovation in these cases covers new forms 
of collective behavior. 

We can find each type i in iV^ exemplars of elements in the system. The 
exemplars may be individuals, but also organizational and institutional units 
like firms and groups. Ni, the occupation numbers, are functions of time. They 
are positive or zero. A complete set of occupation numbers A^i, A^2, ■ • ■ ,-^s 
at a fixed time, characterizes the occupation state of this system. The time 
dependent change of the occupation numbers is described by the movement 
of this point in the space. The whole motion takes place on the non-negative 
cone K of the space. In this picture we can describe the case, that a type 
i is not present in the system at time t. That means the type i is occupied 
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with the number zero {Ni = 0). We caU a system an under-occupied system 
if we can make the assumption that the sum of the occupation numbers is 
essentially smaller than the total number of the possible elements |2Sj ■ 

In this picture, an innovation is an occupation of a non-occupied type. 
In deterministic systems, zero occupation can only be achieved in the limit 
of infinite time i — ^ cxd , if a sort died out (zero can be a stable stationary 
state) . For finite times i > types cannot arise, if they are not in the system 
at time t — , and present types cannot die out. The situation is different 
if we use the stochastic picture. In stochastic systems the zero state can be 
reached in finite times t. A stochastic description offers the advantage that 
at finite times new sorts (innovations) can arise or die out. Therefore, the 
stochastic description is especially suited for evolutionary processes and in 
particular for innovation processes. 

Let us note here that any innovation will change the taxonomy of types 
in the system . Innovation has to do with uncertainty and its prediction is 
impossible. With the notion of an under-occupied system we escape the prob- 
lem of determining a priori the place or kind of an innovation. Instead, we 
equip the system with a reservoir of possible innovations. Which of these pos- 
sibilities turns into a realization remains uncertain. In some respects this is a 
trick to avoid the problem with a changing taxonomy. There are other pos- 
sibilities to escape this problem, e.g., so-called continuous models operating 



on a characteristic space as we discussed elsewhere |H4i:^:^l3()| . However, the 
discrete approach has, in our respect, certain advantages, as we will discuss 
later. 

In an under-occupied system most elements have, at a given time t, the 
occupation number zero. So we can pass from the high-dimensional cone K 
to a lower-dimensional cone K^. Accordingly, the time dependent variation 
of the system can be described as a switching of the state point on the edges 
of the cone K. If K~^ is an element of the set of all possible cones, we observe 
a switch from one sub-cone to another. As the process is discrete, it is a 
hopping on the edges of different positive cones. 

In figure n we visualize such a process for three dimensions. At any point 
in time, the state of the system is represented by a certain vector N{t). In 
a stationary state, the endpoint of this vector defines a positive cone. In our 
example, the vector moves in the plane spanned by A'^i and N2. An innovation 
opens up a new dimension of the system. In our example, a new third type 
is introduced into the system. After the innovation, the vector is moving in 
the space defined by A^i, A^2 and 7V3. In general, we can assume that the 
system operates in a multidimensional space where the cone can have a very 
complicated shape, and the vector N{t) jumps between the edges of this 
cone. 

The hopping process visualizes the transition between one stable station- 
ary state and another stable stationary state. In this sense, innovation is the 
outcome of a process of destabilization. In the framework we propose in this 
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Fig. 1. The occupation number space: The distribution of individuals over different 
types is represented by a vector in the positive cone. The appearance or extinction 
of a type can be described as approaching or leaving one of the edges of the positive 
cone. 



paper, innovations are seen as stochastic instabilities. The changes occur- 
ring in the occupation number space result from interactions of the different 
types present in the system. These interactions can be visualized as graphs or 
networks where the nodes represent the types, and links between them rep- 
resent different forms of interactions. In the network picture, an innovation 
corresponds to the appearance of a new node and the activation of a link to 
this node. The models we will present in section 3 allow us to differentiate 
between different processes which finally introduce such a new node. 

The conceptualization of types as elements (nodes) of a network represents 
a graph theoretical approach to the dynamics of the system. Therefore, other 
network approaches are of particular relevance to develop our theoretical 
approach further. We will use the following part of section 1 to introduce the 
new emerging specialty of complex network theory and to place our approach 
in this field. 



1.2 Innovations, the Emergence of the Field of Complex Network 
Theory and Sensitive Networks 



In the last years, complex systems in nature and society have been carefully 
investigated. Already in the 70s, theories of self-organization were used to 
build a bridge between social and natural systems investigations |H2] ■ As part 
of this development complex networks have been investigated. Recently, as 
a new branch in complexity theory [ TUBJ complex networks have been recon- 
sidered and extensively studied |1(J5 :. They seem to be particularly relevant 
for the study of innovation processes. 
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In the context of complexity theory, the concept of networks has not only 
been used as an easy-to-use metaphor. As Bornholdt and Schuster note: "Re- 
cent advances in the theory of complex networks indicate that this notion 
may be more than just a philosophical term. Triggered by recently available 
data on large real world networks (e.g., on the structure of the internet or on 
the molecular networks in the living cell) combined with fast computer power 
on the scientist's desktop, an avalanche of quantitative research on network 
structure and dynamics currently stimulates diverse scientific fields." [T^. p. 
V. Social networks form one important area of application of complex net- 
works theory. The structures found cover networks of collaboration 89 tl, 
networks of recognition (citation networks) jl23| and networks of corporate 
directors |119j . In economic theories, innovation is more and more understood 
as the outcome of the interaction between scientific, economic and political 
systems ^Q. Instead of considering an innovation as a singular event the 
network character of innovations is stressed. Innovation networks seem to be 
a new organizational form of knowledge production. 

The structural analysis of systems is of great interest. Albert and Barabasi 
12 give a very good presentation of this subject and its development. In 
the very beginning, investigations of large complex systems were done by 
random graphs. More and more it became possible to analyze real complex 
systems and large systems too. With the development of computer capacity, 
the amount of empirical data increases. It becomes possible to compare the 
theoretical results by random graph theory with that of the real data analysis. 
Obviously more than pure randomness exists. Organization principles and 
rules of system evolution play a decisive role, leading to small-world behaviour 
and scale- free networks. Our world is not a random world. Other evolutionary 
principles are of great interest. In addition, it is evident that a theory of 
evolving networks may give a more realistic approach to real systems. This 
is why we give special consideration to evolving networks here. 

From the analysis of empirical data we learn that many real-networks 
have a small- world character. The small- world concept describes the fact 
that despite their often large size, in most networks there is a relatively short 
path between any two nodes. The small- world property characterizes most 
complex networks. For example, the chemicals in a cell are typically separated 
by only three reactions, or in a more exotic case, the actors in Hollywood are 
on average within three co-stars from each other. The small-world concept 
corresponds to our observations; it is a structural, not an organizing, principle 

inzEHi. 

Not all nodes in a network have the same number of edges (the same node 
degree) . The spread in node degrees is characterized by a distribution function 
P{k), which gives the probability that a randomly selected node has exactly k 
edges. Since in a random graph the edges are placed randomly, the majority of 
nodes have approximately the same degree, close to the average degree of the 
network. The degree-distribution of a random graph is a Poisson distribution 
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with the peak over the average degree [2]- In real networks the distributions 
of the edges are more complicated. Important results were obtained by the 
analysis of large real systems: the degree distribution deviates significantly 
from a Poisson distribution and follows general structural rules in many cases. 
Many large networks are scale- free, that is, their degree distribution follows 
a power law. In addition, even for those networks for which P{k) has an 
exponential character, the degree distribution significantly deviates from a 
Poisson distribution achieved by random graph theory for such systems |2I()| . 

Scale-free networks express a hierarchy between the nodes. Not every node 
is important at the same level. Accordingly, not each of the links between the 
types have the same importance. There are very sensitive relations or ele- 
ments too. As mentioned previously, we consider networks as "sensitive" if 
their properties depend strongly on the introduction or removal of one or a 
few nodes or edges, or on changes of the occupation of nodes. We will show 
that for the evolutionary character, the description of the time-behaviour by 
master equations on occupation number spaces is an appropriate tool. The 
discrete character of occupation number description allows for an appropri- 
ate description of the introduction respectively of removal of relations, edges 
etc.. We will analyze not only the steady states of our stochastic systems 
but also the time evolution. Albert and Barabasi 12 also refer to approaches 
with master and rate equations. They write that in addition, these methods, 
not using a continuum assumption, appear more suitable for obtaining ex- 
act results in more challenging network models. In addition they mentioned 
that the functional form of the degree distribution, P{k), cannot be guessed 
until the microscopic details of the network evolution are fully understood. 
According to our point of view, the method of master equations is an excel- 
lent tool to use in the investigation of many open questions and is able to 
bring much more light to bear on this subject. For example, by using this 
discrete approach, we have the chance to get statements about the kind of 
fluctuations. Evidently this is one of the most important questions. 

Let us come back now to the question of the distribution of the graph. We 
remember that random graph theory leads to a Poisson degree-distribution. 
Albert and Barabasi j2| give a near exhaustive survey of empirical data sets 
for real complex networks and show that the real degree-distributions are 
not Poisson-distributions, but scale-free-distributions, or exponential distri- 
butions. These authors write: "The high interest in scale-free networks might 
give the impression that all complex networks in nature have power-law de- 
gree distributions. ... It is true that several complex networks of high in- 
terest for scientific community, such as the world wide web, cell networks, 
the internet, some social networks, and the citation network are scale-free. 
However, others, such as the power grid or the neural networks of c. elegans, 
appear to be exponential . . . Evolving networks can develop both power-law 
and exponential degree distributions. While the power-law regime appears to 
be robust, sublinear preferential attachment, aging effects, and growth con- 
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straints lead to crossovers to exponential decay. ... If all processes shaping 
the topology of a certain network are properly incorporated, the resulting 
P(fc) often has a rather complex form, described by a combination of power 
laws and exponentials." 

In this respect our aim here is the calculation of the role of fluctuations 
by the use of the master equation approach. This way we can make state- 
ments as to how the systems differ from linear systems which obey a Poisson- 
distribution. In principle, by investigating the fluctuations, correlations and 
spectral-densities, we are able to study several microscopic events. One of 
the questions to solve is which fluctuation effects produce power-law dis- 
tributions. We see a deep connection of these network systems to systems 
which produce l//-noise. We remember that l//-noise is a stochastic pro- 
cess with a specific power-law spectrum |7()| . A characteristic property of 
processes which produce l//-noise are long range-correlations. We suppose 
that the scale-free networks and small-world-behaviour may have some rela- 
tion to this. We remember that in small- world networks the degree- function 
obeys an power-law; there exists a small pathway between each two of the 
elements. 

In investigating selforganization and evolution processes in networks, our 
basic approach is that we understand the corresponding networks as dynamic, 
or more precisely, as evolutionary systems. This dynamic and evolutionary 
approach allows us to make statements about innovation processes, special 
competition effects, the sensitivity of networks, the constraints of growth and 
the fitness of network systems. 

2 Structure and Relations of Socio-Economic Networks 

2.1 Structure, Selforganization and Complexity 

With respect to their structure, social connections are relations between ele- 
ments. Therefore, as can be found in each handbook of social network anal- 
ysis, relational data form the basis of social networks |111I126| . Such data 
describe ties, connections, group attachments, meetings and other events or 
activities that relate one individual to another one. From the abstract point 
of view, socio-economic networks are "structures" . It is possible to give an 
abstract presentation of such structures by means of mathematical tools. The 
mathematical idea of structure stands in close connection to the terms 

element, set, relation and operation. 

The nature of the elements does not play any role with respect to the struc- 
ture. Opposite to this, the nature of the relations between elements deter- 
mines the specificity of a structure. Structure here means the manifold of 
interactions between the elements. 

Let us note here, that for many descriptions of socio-economic networks 
as in sociology and economics, it is just the nature of the elements and the 
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relations which are relevant. In this paper, we follow a more abstract and 
general approach. By using such a general level, we create methodologically 
the task of re-specifying the definition of both elements and relations for any 
application area that one might think of. On the other hand, the high level 
of generalization we use here keeps the application areas open, as we intend 
to do. 

The idea of "structure" is of great importance in our life - both in reality 
and in sciences. In particular, it is obvious by analyzing social structures. 
What does "structure" mean in the original sense? On the one hand we have 
our conventional understanding of structure - the understanding of structure 
in our real life. On the other we may formulate a precise idea of structure in 
terms of mathematics, in system theory 21 and in the theory of selforgani- 
zation |29j : 

"We understand under a "structure" the composition of elements and the set 
of relations respectively operations, which connect the elements." 

Krober [Jlj writes about the idea of structure in real systems: "Each system 
consists of elements that are arranged in a certain way and are linked to 
each other by relations. We understand by 'structure of a system' the kind of 
arrangement and connections of their elements ... In this respect, we do not 
consider what kind the elements are. If we speak about structure, we are not 
interested in the elements of the structure. We only consider the manifold of 
relations. In this respect the structure of a system is a well-defined connection 
between the elements of the system. These elements, which are arranged 
in a determined manner and connected by determined connections can be 
necessary or randomly, universally or uniquely, relevant or irrelevant" . 

Moles |H21 writes to this subject: "The surrounding objects of the material 
world, artifical and natural organism in the wide sense of this world are signed 
by two main aspects; by their structural and functional properties." 

With the famous book series "The Elements of Mathematics" the group of 
scientists, Bourbaki, gave an example for constructing systematically math- 
ematics as a science of such "structures" . In the following section we give a 
short description of several important concepts of this mathematical theory 
of structures. In particular, we give a summary of the theory of relation, 
graphs and matrices in the amount we need here. Our purpose is to apply 
this abstract theory to socio-economic networks. 

In a socio-economic system, the elements of structure are individuals or 
groups of individuals in different institutional and organizational forms, e.g., 
firms. The socio-economic connections between these elements are relations 
in the sense of this abstract theory of structure. The description of elements 
and relations can be given graphically by a system of vertices (nodes), which 
model the elements (individuals, groups, firms) and of edges (arcs), which 
describe the relations (connections). Nodes and arcs can be weighted. Arcs 
can have a direction. This way we can include quantitative aspects. The 
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most important aspect in the selforganization of networks is the formation 
of new connections, which generates new structures. In the foUowing, we 
want to show that the instruments of the mathematical theory of structure 
in connection with the ideas and concepts of the theory of selforganization 
contribute to the description of the connections in complex socio-economic 
networks. 

The idea of "structure" stands in a close connection to order (disorder) 
respectively information (entropy) . The theory of selforganization shows how 
the generation of structures is connected with the decrease of entropy [221 ■ 

Socio-economic systems are complex systems, which consist of many con- 
nections between the elements. Therefore, complexity is a further concept to 
be defined. Ebeling, Freund and Schweitzer :29; write: 

"As complex we describe holistic structures consisting of many components, 
which are connected by many (hierarchically ordered) relations respectively 
operations. The complexity of a structure can be seen in the number of equal 
respectively distinct elements, in the number of equal respectively distinct 
relations and operations, as well as in the number of hierarchical levels. In 
the stricter sense, complexity requires that the number of elements becomes 
very large (practically infinite)." 

We are especially interested here in the origin of complex structures, and in 
the development of order (information). In the end we have to answer the 
question which parameter-relations (order parameters) determine the quali- 
tative behaviour of the system. Ilya Prigogine [HJ in collaboration with his 
coworkers, did pioneering work in the investigation of selforganizing systems. 
I^U] Further important work has been done in this field by Manfred Eigen 
\IM^\ on the selforganization of macromolecules and by Manfred Eigen and 
Peter Schuster |H8I4()| on the hypercycle model. The mechanisms of selforga- 
nization are clearly worked out by Nicolis and Prigogine 1^ . These authors 
give a stringent physical and mathematical formulation for these processes, 
in particular with respect to the energetic and entropic aspects. A somewhat 
different view on this was developed in the formulation of the synergetics 
by Haken |52j . The investigation of such systems shows that the formation 
of order in complex systems can be allocated to physical processes which 
play a role far from equilibrium [27]. We underline that biological, just as 
socio-economic processes, can be investigated with the help of the theory 
of selforganization because they obey the valid physical and chemical laws. 
However, processes which include the real life (biological and socio-economic 
systems) also obey other rules and laws, which are not solely determined by 
physics. This is already evident from the very general character of the struc- 
tures we consider here. As said above, we formulate the idea of structure 
mathematically and keep away from how the structure appears in different 
systems. Then, graph theory will provide us with the level of abstraction we 
need in order to describe real systems. 
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2.2 Introduction to the Theory of Relations, Matrices and 
Graphs 

By formulating our ideas in the mathematical language, we have the advan- 
tage of having access to the great mathematical potential which is available 
in this field. An important basic assumption is to start from the theory of 
sets. As pointed out above, the connection between element and set is the 
first and most important aspect of a structure. Furthermore, we introduce 
relations and operations. The concept of structure reflects abstract proper- 
ties of a system. Due to the abstract character of the concept, results can be 
translated to other systems and comparisons are possible. 

In general we can differ between local, temporal, causal and functional 
structures. To illustrate the structures we will use graphs which represent 
the elements and their connections by geometrical symbols 55 74 21 27 451. 
An example of a graph representing an economic network with 4 levels is 
given in Fig. |2] This graph represents the flow of materials and outcomes in 
a production process. 

In economics, the relations between economic agents can be represented 
in a network form. In this case information flows, e.g., price signals between 
market participants, are exchanged. Then, the structure of the network de- 
scribes a situation with local interaction (not every agent is informed about 
all other agents but the agents also do not act independently) |SH]. Socio- 
economic networks can also describe a variety of different agents' actions 
that influence other agents. The diffusion of technologies over firms can be 
described as a network of actions from formation of a company (with a certain 
technology) over knowledge transfer between companies (in form of imitation 
or merging) to the exit of companies due to technological competition. We 
will come back to such a network interpretation in section 3. 

It may be worthwhile to underline that it is not possible to give a complete 
structural description of living objects by binary relations as graphs only. 
Graphs are just a device for analyzing these systems, albeit a very useful 
tool. Of course, we cannot describe complex objects only by graphs because 
their binary relations are ambiguous [57]. Nevertheless, graphs are very useful 
for the representation of complex structures. Before the elements of the theory 
of graphs are explained it is necessary to introduce the two notions; set and 
relation. 

What we understand by a set is nearly the same as in common language. 
It is always abstract and determined by its elements. A set M is given, which 
consists of the elements oi, 02, 03 • ■ •; symbolically M — {ai, a2, 0,3 ■ ■ -{.The 
elements a.i {i = 1, 2, 3, • • •) describe individuals or groups equipped with dif- 
ferent attributes and features and belonging to different types. The number 
of elements determines whether we have a finite or infinite set. Our investiga- 
tions here are related to finite sets. In the following "sets" means finite sets. 
Graphically the elements of a set can be described by vertices (nodes) in a 
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Wj* 4. level: Level of consumers 



3. level; Producing final goods 



2. level: Intermediate products 



1. level: Raw materials 



Fig. 2. Graph of an economic networlc with 4 levels: (i) raw material such as min- 
erals, fossils, plants etc., (ii) intermediate products as steel, coal, corn, etc., (iii) 
producing final goods, (iv) level of consumers 



n-dimensional space. This way we assume that a clear relation exists between 
the vertices and the elements of the set. An example is given in Fig. |3 



ai 



02 



03 



M = {01,02,03} 



Fig. 3. A set of three elements (for instance, plants or firms) which are represented 
in a one dimensional space. 



The elements of a set can be ordered to pairs. Let us consider for example a 
set M — {fli, 02, . . . , 05} with the pairs [ai, as] and [02, 05]. ai and a2 describe 
the first element and 03 and 05 the second one. The pair [05,02] describes 
another relation between the elements 05 and 02 as [02,05]. In general the 
elements Oi and Oj in an ordered pair [oi, aj] are not exchangeable with each 
other without changing the kind relations of the elements a^ and aj to each 
other. 

As an example we consider a set of firms with a certain economic struc- 
ture between them. The structure can be determined either in the form of 
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trade relations, financial transactions or information flows. Usually, relations 
in an economy are directed. The set M consists of three plants ai, 02 and 
03. There are given the following relationships: oi provides certain products 
to a2 and to 03; 02 delivers products to 03. The ordered pairs for these rela- 
tion follows to:[ai, 02], [ai, 03] and [02, 03]. We can represent the relationship 
geometrically. This representation is called a directed graph (Fig.0}. 




M = {01,02,03} 

Fig. 4. Directed graph 

A graph consists of the elements of the set and the relations. In such a way a 
graph can give determined relationships between elements of a set. In general 
a graph is a structure model which mirrors aspects of the structure of the 
investigated object. In order to obtain this structure we have to consider the 
relations between the corresponding partial objects. Let us note, that both 
the notion "vertex" (vertices) and "node" is used to describe the elements. 
Their relations are represented by "edges" or "arcs". Sometimes, the notion 
of an "arc" is used for an directed arc only. Other synonyms arc link or line. 
We consider a system to be a certain representation of a real phenomenon 
where certain boundaries between the system and its environment can be de- 
fined. A system is, in general, defined by the set of its elements and the set of 
relations between the elements. Obviously, with regard to the original phe- 
nomenon, different degrees of adjusting the model to the origin are possible. 
If a unique map exists between the structure of the original phenomenon and 
the structure of the model, the model is called homeomorph. The border line 
case of maximum adjustment of the model to the original in respect to the 
structure is called isomorph. 

2.3 Structures with Random Relations and Their Application to 
Socio-Economic Groups 

In the previous subsection we represented the individual units (e.g., plants or 
firms) as elements (vertices) of a graph. The relation of the units are shown 
by the edges. Let us introduce several simplifications. First we exclude the 
relation of a node to itself (self-reflexibility) from our consideration, as this 
leads to a special type of graph. Further we only consider graphs with dis- 
tinguishable vertices and indistinguishable directed edges. A directed graph 
is also called a digraph. We exclude loops and parallel edges. We concentrate 
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in this section on random graphs. The capacity of a network (graph) - the 
number of its elements - is one of its important functions. Of relevance for 
the structure is also the connectivity of a network. Already 1970 Gardner and 
Ashby |51j investigated the probability of the stability of great networks, in 
dependence on capacity and connectivity. May |Hl] performed similar investi- 
gations. Computer simulations and some statements of probability about the 
structural behaviour of networks may be found also in papers of Sonntag, 
Feistel, Ebeling |118j and Sonntag jll(i| . We now introduce several impor- 
tant terms that characterize a network, following in part the work mentioned 
above. 

If K is the number of edges in the graph, and S is the number of vertices, 
we denote the considered graph by D{S,K). As a simple and important 
measure, the connectivity is defined as: 

Isolated, connected parts of the digraph D{S, K) consisting of s vertices 
and k arcs are called components d{s, k). Let us note here that the "basic step 
in the structural description of a network is to identify the number and size of 
its components" |111| . p. 102. Socio-economically interpreted, the existence, 
number and size of components in a graph stands for the opportunities and 
obstacles to communicate, to exchange information or/and to interact. 

Sf. is the number of components with s vertices and k arcs. So we can 
write: 

Y^sSi ^ S- Y.^^k ^ K (2) 

s,fc s,k 

We denote the mean values of the frequency of the component d{s^ k) by: 

Hi = {SD = Y. ^ Pk,r, (3) 

r 

with P^^ being the probability that in a special digraph D{S,K) S^ — r 
components d{s, k) can occur. 

As mean number of components of the digraph D{S, K) we use: 






^ = YYHl (4) 

s A; 

Apart from the trivial components, such as single vertices {d{l, 0)) and single 
arcs (d(2, 1)), the digraph contains a "structured part", whose number of 
components is written as follows: 

i = 77 - ffi - Hi (5) 

The probability distribution P^^ is not known for the finite digraphs. In 
the limit S -^ oo the probabilities for the emergence of different finite large 
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components are uncorrelated, so we have to work with a Poisson distribution 
for the possibility of r components with s vertices and k arcs in the whole 
graph [TT^ : 



lim Pt 

S->oo " 



{Hff 



exp (-i?|) for 5* > s. 



(6) 



2;r 



0.1 - 



1 ^ 

1 

Fig. 5. The probability distribution P|^ for C = 0.05 that r components d(3, 2) 



In dependence on C we can for instance represent P^^, Pf^, P|^, P|,, 
over r. So we can show the building of components in dependence on the 
connectivity C (Fig. |SJ|. 

Following Sonntag, Feistel, Ebeling |118| we can achieve further state- 
ments. In the limit 5' — > oo the mean number of components rj in practice 
is determined only by semicycleless components (trees). So we can use the 
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approximation: 

ri = E ^-1 + 0{S') (7) 

In that limit 6*^00 the number of non-trivial components: 

L = T] - S exp(-2C) - SC exp(-4C) (8) 

follows. 

By a constant number of S of individuals, firms (vertices) the building 
of relations between these individuals, firms corresponds to an increasing 
number K . K is proportional to C : 

C = — r^ K ; S ^ const. (9) 

In such a way, and intuitively clear, the connectivity C increases during the 
development of relations. We will show that firstly the number of small com- 
ponents increases and then decreases with increasing C . Great components 
become important. In the end all components are "absorbed" into one great 
component. 

In our case, great and strong connected components d{s, k) are of interest. 
This means that a great number of relations exists fc > s, so that the compo- 
nents are strong connected. Strong connected components are characterized 
by the appearance of cycles. For S* -^ 00 the frequency of cycles of the length 
I in a graph D{S, K) is: 

H^^ = ^ (10) 



and for semicycles with the length I 



He - ^^ (11) 

The number of vertices of D{S^ K) belonging to cycles is: 

Zc = C^ (1 - C) for 5 ^ 00; C < 1/2 (12) 

and of those belonging to semicycles 

Zo = 4 C^ (1 - 2C) for S* ^ 00; C < 1/2. (13) 

The number of vertices of D{S,K) belonging to one tree in D{S,K) re- 
ferred to S is equal to the value given by Erdos and Renyi P2j : 

r ^^-^^ /I C<l/2 

y ^ tlo ^ = \x{C)/2C 01/2 (14) 
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with 

s 

rk,s = E ' ^r' (15) 

a; e~^ = 2 C e'^C (^g) 

OO s— 1 

^(^)-E V (2^^~^^)' (17) 

The graphical representation shows analogies with a phase transition of 
2nd Qj.(jgj. _ g^ fg^(,-(; -j^jjg^-j; becomes particularly obvious if such networks are 
considered by means of the percolation theory |117| . For C < 1/2 the proba- 
bility is zero, that from a vertex a cycle goes out. For C > 1/2 the probability 
increases up to the value one. Opposite for C < 1/2 a vertex belongs with 
the probability one to a tree. For C > 1/2 that probability goes to zero. 

For C > 1/2 the graphs D{S,K) consist of one great component. For 
C > 1/2 with growing C value, one tree after the other is "absorbed" by 
the great component. With an increasing number of arcs ever more trees are 
linked up with one great component until, finally, all vertices are linked with 
each other (Fig. Ej) . 

The number of components in directed graphs also corresponds to the 
value given by Erdos and Rcnyi |22] for undirected graphs (Fig.[7|): 

S fl-C- 0<C<l/2 

^ = i™„ - = S ±. (^(n\ _ £!M^ c > 1/2 (^^) 



For the examination of the development of socio-economic connections, 
it is interesting to inspect the share of the graph, that is structured. If we 
consider a network build from firms the number of the isolated vertices (firms) 
can be determined. In particular, they are those which do not belong to the 
connection net. In the same way, the number of components (without isolated 
vertices) can be obtained. This is the part of the graph that includes the 
economic or knowledge exchange network. 

If we understand economic connections as a manifold of possibilities, we 
can arrive at some conclusions. In particular, we will observe the network of 
interaction between firms over time. We keep the number of firms (vertices) 
in the network constant and let the connections between them develop, i.e. 
K, the number of edges, will increase. On the other side K is proportional to 
C. In accordance to intuition, the connectivity rises if connections develop. 
As the calculation shows, at first the number of the little components rises 
and then it decreases with the further rising of C. The number of the great 
components rises up to the moment when all are absorbed in one. Inside 
this great component the degree of connectedness of the network plays a 
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Fig. 6. The relative number of vertices of D{S, K) belonging to a tree {y) in de- 
pendence on the connectivity |42| 



role. A sign for this strong connectedness is the occurrence of cycle in the 
graph. As we have shown, the frequency of cycles of the length I is a simple 
function of the connectivity C . Likewise, the probability can be given that a 
cycle comes from a vertex. For C < 1/2 the probability is zero, that means 
that the number of the vertices is unimportant. By a rising development of 
connections (C > 1/2) this probability rises up to 1. 
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0,5 



Fig. 7. The relative number of components 5 of a graph in dependence on C 



These findings have consequences for the development of information flows 
in growing socio-economic networks. The question is if a certain type of devel- 
oping connections can also be observed in empirical data about information 
flows. For instance, inside or between organizations of firms. One possible 
method to trace information inside of organizations is the use of e-mail traf- 
fic. In a recent paper Tyler, Wilkinson and Huberman |122| used e-mail data 
to map the communication network inside an industrial organization. In this 
case the nodes represent people and the connections between them are given 
by e-mail correspondence. They identified components of different size and 
interpreted the smaller components as expressions of communities in prac- 
tice. As far as they reported, the structure of the e-mail networks shows 
similarities to the structure of the organization. In addition, leadership roles 
can be identified this way. To relate such empirical observations to the re- 
sults repeated above, one would need to look at the temporal evolution of 
components and connectivity in time. 

Another recent approach using percolation theory to describe socio-eco- 
nomic change was proposed by Silverberg and co-authors |113I112| . Starting 
with a space of discrete technologies which have a certain performance, they 
looked for spanning connected paths in this technological landscape. In this 
picture, invention can be visualized as isolated islands located ahead of a 
technological frontier which moves forward. Under the critical value there 
will only be finite connected sets (clusters or components) of different tech- 
nologies, and technological change might come to an end. If we remember that 
firms are the carriers for different technologies, there is a direct link to the 
concept of a network of firms exchanging information about different tech- 
nologies and doing search processes in such an abstract technological space. 



20 Hartmann-Sonntag et al. 

Then, the different components would be related to groups of firms sharing 
certain technological knowledge or being part of one chain of production. The 
increase in connectivity characterizes the diffusion of a certain technology in 
a specific industrial sector or across sectors. 

If we consider a firm, and do not look at the whole graph, additional 
aspects can be included: If we understand the concept of the economic con- 
nections as a sum of abilities which the firm has acquired, we can write in 
simple terms: 

competence = ^ of the abilities 

If a firm or plant creates a new economic connection, it can be in interaction 
with others. In the picture of the graph it will be represented, as mentioned 
previously, through a chain from i to j. 



aij 



Fig. 8. Edge, which goes from i to j 



The firm i builds a connection to firm j. The connection from i to j is 
described by atj. The elements atj build the adjacency matrix. If the sum of 
economic connections from i — J^j '^ij '^^^ be represented in a specific way 
we speak of firm i as a source in the network (Fig. ^ . The sum of connections 
corresponds to the sum of arcs which leave the vertex i. This is exactly the 
sum over the elements of row i of the adjacency matrix A. 



k 



n 

Fig. 9. Source 



Connections which related a set of other firms to the firm i can also be 
illustrated in the graph-picture (Fig. I10() . The sum of these connections is 
equal ^ ■ ajt. This corresponds to the sum of the elements of the column i of 
the adjacency matrix. 
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m» 



Fig. 10. Sink 

Connections which are based on mutual relations at different times, can be 
described in the graph-picture by establishing two arcs between two vertices 
(Fig.CH). 





h 



Fig. 11. Mutual connections 

In that picture the sum of these connections is proportional to the sum of 
the cycles which leave the vertex i. So we can get the number of cycles of the 
length / from the Ith power of the adjacency matrix AK 

Finally let us underline that this is just a very small part of the network 
theory which we explained here. We concentrated on topics relevant to our 
task of describing selforganization and evolution processes. 



3 Innovation Processes and Other Stochastic Effects in 
Networks 

3.1 Overview of Stochastic Effects in Networks 

3.1.1 Birth and Death Processes A special stochastic process called 
"birth and death process" is of particular importance to our study of stochas- 
tic effects in economic processes, and in particular in innovation processes. A 
birth process is a random appearance of a new element in a system. A death 
process is the disappearance of an element. Processes of this kind play a big 
role in biology, ecology and sociology. 

Processes of this type are also relevant to the field of economic processes 
|(i()| . Economic growth is characterized by structural changes based on the 
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introduction of new technologies in the economic world. To describe techno- 
logical evolution, one has to determine the system, the elements and their 
interactions. Here we consider plants and technologies. We consider firms 
composed from different plants. The plants are introduced as elementary 
units which play the role of decision carriers according to market conditions 
(choosing a new technology or not). Plants also play the role of users of a 
particular technology. The technologies are understood as the different types 
present in the system. The plants are the elements or representatives of these 
technologies. In this perspective, eventually technologies are competing for 
plants using them. This perspective differs from the way one usually thinks 
about technological change, where the firm is central. The underlying pro- 
cess is still a decision made by plants or firms. However, the model approach 
constructs an inverse perspective on it. Let us note here that this perspec- 
tive is quite normal for any population dynamic approach which deals with 
types (groups or species) and elements (individuals). However, in contrast to 
biological processes, human beings, organizations and firms are not bound 
to a certain type or group they first belong to. In contrast to individuals of 
biological species they have the opportunity to change the group they belong 
to. It is this kind of transition behaviour that makes the model particularly 
relevant for socio-economic applications. Let us note further that we delib- 
erately use the notion of a plant or production unit as the simplest element 
in the system. By assuming that firms consist of several plants or produc- 
tion units, growth processes of firms are also covered by the model approach. 
Technological change is usually considered as a macroeconomic change pro- 
cess. However, in order to describe it as an evolutionary process, one has to 
consider this process at the microscopic level. This means we have to con- 
sider the microeconomic carriers of technological changes. In the framework 
we present here, these are the plants \W\ . 

The basic ideas for the modelling of these processes go back to so-called 
urn-models. Already in 1907 the physicists Paul and Tatyana Ehrenfest de- 
veloped a simple model for the diffusion of N molecules |SZ1- The Ehrenfests 
studied two urns, A and B, which were isolated with respect to exchange with 
their surroundings. With respect to exchange between urns the Ehrenfests as- 
sumed permeable connections between A and B. Because of the isolation of 
the two urns, the total number of molecules in A and B remains constant. At 
regular time-intervals, a molecule is randomly (that means with the proba- 
bility (1/A^)) chosen and changes from its urn to the other urn. 

In 1926 Kohlrausch and Schrodinger [JI] gave a continuous diffusion- 
approximation for such processes. Feller |48| formulated a realistic variant 
of this model. He used a discrete Markovian process with continuous time. 
The time between the molecule crossings was exponentially distributed. 

Originally developed for molecular processes, the model soon found many 
apphcations to biological processes. Surveys of biological apphcations of birth 
and death type processes were given by Bartholomay J8l9ll()j and Eigen p^ . 
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The model of Ehrenfest represents the prototype for the investigation of 
decision processes in a group between the possibilities A and B respectively 
between yes and no PJ- For example, we may consider the decision to accept 
a new technology or not. Applications to social and economic processes were 
surveyed by Weidlich |129| . 



3.1.2 Stochastic Effects in Small and Sensitive Netv^^orks Many 
complex systems display a surprising degree of tolerance to errors. The results 
indicate a strong correlation between robustness and network topology. In 
particular, scale free networks are more robust than random networks against 
random node failures, but are more vulnerable when the most connected 
nodes are targeted 0]. 

In small networks any nodes or edges play a specific role. Their addition 
or removal drastically changes the properties of the whole system. Another 
problem where stochastic effects play a big role is the question of how a single 
new mutant can win the selection process. If one considers networks of web 
sites and competition processes between them about attracting visitors one 
would ask how can a new web site become a giant cluster among other already 
important web sites (clusters)? Is it possible to overcome the "once- forever" 
selection behaviour by stochastic effects? 

The addition or removal of sensitive nodes or edges is a subject of inves- 
tigation in big networks. With sensitive we mean here elements which play a 
special role in the network. Let us refer to some examples. 

Cellular networks can be subject to random errors as a result of muta- 
tions or protein misfolding, as well as harsh external conditions eliminating 
essential metabolites. Jeong et al. ^J studied the responses of the metabolic 
networks of several organisms to random and preferential node removal. Re- 
moving up to 8% of the substrates, they found that the average path length 
did not increase when nodes were removed randomly. However, it increases 
rapidly after the removal of the most-connected nodes and up to 500% when 
only 8% of the nodes are removed. Similar results have been obtained for the 
protein network of yeast as well |63I124| . 

Sole and Montoya |115| studied the response of the food webs to the 
removal of species (nodes) (see also |12]). The results indicate that random 
species removal causes the fraction of species contained in the largest cluster 
to decrease linearly. However, when the most connected (keystone) species are 
successively removed, the relative size of the largest cluster quickly decays. 

The error and attack tolerance of the Internet and the World Wide Web 
was investigated by Albert, Jeong and Barabasi j^. The Internet is occasion- 
ally subject to hacker attacks targeting some of the most connected nodes. 
They show that the average path length on the internet is unaffected by the 
random removal of as many as 60% of the nodes, while if the most connected 
nodes are eliminated (attack), the average path length peaks at a very small 
fraction of removed nodes. Albert, Jeong and Barabasi investigated the World 
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Wide Web jSj and showed that the network survives as a large cluster under 
high rates of failure, but under attack, the system abruptly crashes. These 
authors write: "The result is that scale-free networks display a high degree of 
robustness against random errors, coupled with a susceptibility to attacks." 

Wagner and Fell |125| studied the clustering coefficient, focusing on the 
energy and biosynthesis metabolism of the Escherichia coli bacterium. They 
found that in addition to the power-law degree distribution, the undirected 
version of this substrate graph has a small average path length and a large 
clustering coefficient. 

Bianconi and Barabasi [12j showed the existence of a closed link between 
evolving networks and an equilibrium Bose gas. According to them, the map- 
ping to a Bose gas predicts the existence of two distinct phases as a function 
of the energy distribution. In the ffi-get-rich-phase, the ffiter nodes acquire 
edges at a higher rate than older but less fit nodes. In the end, the fittest 
node will have the most edges, but the richest node is not an absolute winner, 
since its share of the edges decays to zero for large system size. 

Maurer and Huberman |79j present a dynamic model of web site growth 
in order to explore the effects of competition among web sites. They show 
that under general conditions, as the competition between sites increases, 
the model exhibits a sudden transition from a regime in which many sites 
thrive simultaneously, to a "winner takes all market" in which a few sites 
grab almost all the users, while most other sites become nearly extinct. This 
prediction is in agreement with empirical data measurements on the nature 
of electronic markets. 

Dorogovtsev, Mendes and Samukhin |22] developed a statistical mechanic 
approach for random networks. They summarize: "Using the traditional for- 
malism of statistical mechanics, we have constructed a set of equilibrium sta- 
tistical ensembles of random networks without correlation and have found 
their partition function and main characteristics. We have shown that a 
"scale-free" state in equilibrium networks without condensate may exist only 
in a single marginal point, so that in such an event this state is an exhibi- 
tion." They underline the important fact that this differs crucially from the 
situation for growing networks. The latter, while growing, self-organize into 
scale-free structures in a wide range of parameters without condensation. 



3.2 Stochastic Analysis of Innovation Processes 

3.2.1 A General Formulation of the Model The stochastic approach 
given here is based on a model, which was developed in the context of gen- 
eral models of evolutionary processes, and in particular biochemical processes 
| 36I106I27I35| . Later, the model found numerous applications in modelling sci- 
entific evolution |32I18| and technological evolution 15 16 . In this paper, we 
first introduce the model framework in a general form and later concentrate 
on its application to economic innovation and competition processes. 
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In the stochastic picture we use the ideas developed in subsection ll.ll and 
in the subsection !!-{ . 1 . II about the occupation number space. Opposite to the 
deterministic models, the stochastic description offers the advantage that at 
finite times new types (sorts, fields, species, technologies) can arise or "die 
out". 

Let us introduce a set of types numbered by z = 1, 2, . . . , s. We denote by 
Ni(t) the number of elements belonging to a certain type. For an economic ap- 
plication Ni (t) represents the number of plants using the technology i (Note, 
that a type corresponds to an urn in the Ehrenfest problem formulation.). 
These numbers are called occupation numbers. They are a function of time. 
The occupation numbers are positive or zero. 

iV,(i) = {0,1,2,...}. (19) 

Now, the state of the system at the time t can be described by the probability 
distribution of the occupation numbers 

P{Ni,N2,...,N,;t) - P{N;t) (20) 

We consider as elementary processes - processes during which only one oc- 
cupation number can change and as transition processes - processes during 
which at most two occupation numbers can change: 



(21) 



For the time being, we also assume that growth and decline processes of 
the total number of elements in the system are possible. However, let us 
note here that the original formulation of the Ehrenfest model only contains 
transition processes. This is due to the fact that in the Ehrenfest model the 
total number of elements remains constant and only exchange between the 
urns occurs. Decision processes in the model occur as transitions of elements 
between types. 

If we assume that all decisions leading to a change of the set of the occu- 
pation numbers depend mostly on the present state, we can apply the concept 
of Markov process. Then, we can describe the dynamics of the system with 
the help of the master equation. This equation is a balance equation between 
building and reduction processes: 

^^^i^' ^^ = WiN\N') P{N') - WiN'\N) P{N) (22) 

with 

N = {iVi,iV2,...,iV,} . (23) 
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The transition probabilities per time unit that the system turns from the 
state N' to the state N or vice versa are expressed by W{N\N') respectively 
W{N'\N). 

The transition probabilities are supposed as follows |44I27I65I35I57| . 

1. Spontaneous generation 

W^ ( . . . , TV, + 1 , . . . , iV„ . . . , TVfc , . . . I . . . , 7V„ . . . , iV„ . . . , TVfc , . . . ) 

= Af^ (24) 

2. Self-reproduction 

W{...,N, + l,...,N,,...,Nk.,...\...,N,,...,N„...,Nk,...) 

= aW N, + £;« N, (25) 

eI'^ ^ A« + b',^^ N, + Cl^l N,N, (26) 

3. Decay 

M^ ( . . . , TV, - 1 , . . . , iV„ . . . , iVfc , . . . I . . . , iV„ . . . , 7V„ . . . , TVfc , . . . ) 

= eP N, (27) 

Sf ) = Af ) + Sg) N, (28) 

4. Conversion/Transition/Exchange/Mutation 

W(. .. ,7V, + 1,. . . ,7V, - 1, . . . ,iVfe, . . . I . . . ,iV„ . . . ,iVj, . . . ,7Vfe, . . .) 

= -B]^^ Nj (29) 

ii;f ^ = A^f + i?(f iv, + slf iVfc + eg 7v,7Vfe (30) 

with j ^i;k^i;j. 

The coefficients can be differently introduced for special cases. For in- 
stance, they can considered as constant or as being functions of the total 
number of elements and other the system parameters. Later approach intro- 
duces an additional non-linearity to the system. 

The content of the four elementary processes introduced above will be 
quite different according to the nature of the system under consideration. 
For instance, for catalytic networks, self-reproduction may appear as result 
of a process of spontaneous self-reproduction (term related to A^ ) , error re- 
production {Bij ) or catalytic self-reproduction {ClA). The decay can appear 
in the form of spontaneous decay [A] ) and of decay related to catalytic help 
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(Bj ■ ) . Transition or conversion processes in catalytic networks correspond 
to mutation processes with reproduction and ternary reproduction processes, 
which play a role with regard to processes with constant overall particle num- 
ber. 

In the case of technological evolution, self-reproduction appears as a 
growth process of firms expanding their number of production units and 
plants using the same technology. Synergetic effects from the surrounding 
network of firms are supposed to take place when the transition rate also 
depends on the number of firms using another technology. Spontaneous gen- 
eration stands for startups. Decay processes both refer to a decrease of firm 
size (closing of production units) and a closing down of firms. The most inter- 
esting process is related to conversion or transition. Here, the use of another 
technology by a production unit is described. The use of a technology may 
be new for the firm only (firm-specific innovation) or for the whole system 
of firms (system-specific innovation). Triggered by R&D, both invention of a 
technology and imitation behaviour is covered by this process. Also in this 
case, other technologies might influence the decision of a firm for a certain 
technology. The advantage of this type of model is that technological change 
is considered as the outcome of the development of a network of technologies 
and firms infiuencing each other. 

The transition probabilities are formulated generally. As a special case 
we can get from this Ansatz the stochastic equations, which correspond to 
the deterministic EiGEN-model 001 with the condition of constant overall 
particle number 



With the help of the s-dimensional generation function 
F(si,S2,...,s.;i) = J2 ^f ' 4' •■■ sf' P{N,,t) with |s,| < 1 (31) 

N 

we can write the master equation with the transition probabilities as follows: 



Fis;t) = Y. 1^^°^ (^* - 1) F + A^"^ sdss - 1) 



dF 
ds. 



.W .,.._r.9F ^ n) . .. d^F 



A^' sM-^)ir- + Bl> s.s,{s.-l) 



'^" ^''^' ' ds, ' '^ ^'^''^' ' ds,ds, 

+ C^ s,s,sk{s, - 1) „ f ^„ + Af^ (1 - sO 1^ 

+ BS's,il-s.)/^+Afis.-s,)^ 
■^ -^ dsidsj ^ dsj 

+ Biy Si(si - Sj) + Bf^ Skist - Sj) - — -— 
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+ (^Ifk SrSk{s^-Sj) 



gsp 



dsidsjdsk 



(32) 



3.2.2 A Network Representation of the Model In order to relate 
the model to the idea of sensitive networks we now introduce a network 
representation of transition probabilities given in (P^ - lp^HjI . We consider a 
system with s interacting types (sorts, fields, plants). This system can be 
described by a graph in which each element i corresponds to a vertex of 
the number i. We mark the transition probabilities for different processes by 
edges of different type in order to distinguish these probabilities: 



1. Spontaneous generation (simple Ai ' 

innovation) 



2. Self-reproduction 



aPn, 







Error reproduction 



4'^. 



Catalytic self-reproduction 
(sponsored self-reproduction) 



B^-^N^.N, 



cSlN^N,Nk 



a 

i 

o: 



k 



3. Spontaneous decay 



A?^N,^ 



Catalytic decay 



bI^^n^n, 



I 

m 



4. Mutation (innovation) 



Mutation (innovation) with re- 
production 



^N, 



BfN.N, 



clfkN^N,N, 



i 

or 



Mutation without reproduction 



bH^n.n, 
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For under-occupied systems the transition from a non-occupied {Ni = 0) 
state to an occupied {Ni > 0) state is of special interest. We will call 
such a transition an innovation. To describe this in the network picture 
we distinguish two states for one vertex. The states are marked as follows: 



O . non-occupied Ni — 

• . occupied A'^; > 

We also distinguish two states for the edges of the graph. Edges which go out 
from unoccupied vertices (iV^ = 0) are omitted, because they are inactive 
(they cannot work) . Active (working) edges are characterized by the graphic 
representation shown above. For socio-economic networks, non-occupied ver- 
tices stay for new technologies, which have not yet discovered. They can be 
understood as hidden possibilities. The model does not allow for the predic- 
tion of a certain new technology but it can make statements how the system 
handles the appearance of new technologies in general. 

Eventually, we get a graph which describes the whole system at a fixed 
time t. If the colour of a vertex is changed, new connections (elementary 
processes) can flare up or former connections can no longer continue. Pro- 
cesses can spread the elements on the non-occupied vertices or they can select 
them from the occupied vertices. The basic structure of the network is the 
"maximal" graph (all vertices i are occupied, all reactions can work). 

In a network with a small overall number of elements (individuals, orga- 
nizations, plants) {N <C s), a lot of vertices are not occupied. Let us con- 
sider a graph with {N » s) and reduce the overall number of elements to 
a state {N <C s). In general, vertices such as i and j become non-occupied 

by the following processes: Af ^iV„ bI^N^Nj, A[fNj, s'ffN.Nj, b'^^N^Nj, 

Cl^NiNjNk. These are the processes of decay and the processes of conver- 
sion. If only a certain part of the process works because of the small overall 
number, we get a graph with a lot of components. With a decreasing number 
of elements (individuals, organizations, plants) the maximal graph with few 
components develops to a graph with a lot of components 27 . The "minimal" 
graph we can get (maximally decomposed) , consists of self-reproduction pro- 
cesses, spontaneous generation (simple innovations), sponsored innovation- 
processes and sponsored innovation-processes with self-reproduction (self- 
reproducing process). Vice versa, vertices can be occupied by the follow- 
ing processes: spontaneous generation (simple innovation) A^ , error repro- 
duction A^j Nj, mutation without reproduction (innovation without self- 
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reproduction) A^- Nj, and mutation without reproduction but with catalytic 

help (sponsored innovation without self-reproduction) B^^ N^Nj. 

We assume that the processes through which the vertices can be occupied 
are very rare. This means spontaneous generations (simple innovations) and 
mutations have a small probability. After a relatively short time the compo- 
nents are in a local equilibrium (first process). Then, sponsored innovation- 
processes play a role starting from an occupied component which is in equi- 
librium. New components (non-occupied) can be occupied (second process). 
This is a hopping process between components. Certain components die out 
under selection pressure, others survive and growth. 



3.3 Application to Technological Innovations 

3.3.1 Technologies with Linear Growth Rates In the following, we 
consider a system in which the total number of elements is constant. As 
mentioned above, a useful instrument to model such processes is the theory 
of stochastic transitions between urns, established by Ehrenfest. Compared 
with the general model introduced so far in the case of a closed system, all 
elementary processes appear as transition processes. Further, let us restrict 
ourselves to the case of technological evolution. Then, the urns stay for dif- 
ferent technologies. Symbolic spheres travelling between the urns stand for 
plants looking for technologies. 

Ni is the number of plants, which uses the technology i, this means they 
belong to the urn i. This numbers are called occupation numbers: 

N,{t) = {0,1,2,...}. (33) 

First, we formulate a simple model for the binary decision process: 

N = Ni + N2 . (34) 

We assume that during elementary processes the occupation number only 
changes by ±1. This is the so called one-step-process. During transition pro- 
cesses at most two occupation numbers can change in such a way: 

For instance, we assume that Ei is the growth rate of plants using technology 
1. For a new technology 2 the growing rate is E2. We assume: 

E2 > El . (36) 

In this case the technology 2 has a greater growth potential or will grow faster 
than technology 1. We will assume that this is an expression for technology 
2 to be "better" or more suitable for plants. In the stochastic picture the 



Modelling Selforganization and Innovation Processes in Networks 



31 



transition from urn 1 to urn 2 will simply have a higher probability than the 
other way around. So, plants will more often change their technology towards 
technology 2. They will replace the old technology by a better (new) one. 
We write the transition probability for this process as follows: 



T4^+(A^2 + 1|A^2) = E2N1 (^ 



E21 = W+ 



(37) 



In particular, we assume, that the transition probability is proportional to 
the number of plants that use the old technology: 



W+{N2 



Ni 



(38) 



Furthermore, the probability is also proportional to the relative number of 
plants that use the new technology 



W+{N2) 



N 



(39) 



We introduce an additional process of spontaneous change from technology 
1 to technology 2 which is described by the coefficient -621. The opposite 
spontaneous transition appears with the coefficient -E12. In total the opposite 
transition process has the probability: 

iVi 

N 



W-{N2 - l\N2) ^ El N2 ['-^] + E12 ^ W^^ (40) 



We can formulate the master equation for this discrete process. This equation 
describes the time-behaviour of the probability distribution of the occupation 
numbers. 

P(7Vi,7V2;i) (41) 

With the mentioned transition probabilities follows: 

^P(7Vi,7V2;t) - W^^_^{N2\N2 - 1) P{N2 - l;t) 

+ VK„,+i(^2|A^2 + l)P(A^2 + l;0 

- VK+^(iV2 + l|iV2)P(A^2;i) 

^W^^{N2-\\N2)P{N2A (42) 

and 



-P(iVi,7V2;i) 



£^21 



^ {N2 ~ 1){N, + 1) 



P{Ni + l,N2-l;t) 
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Ei2 + |r (^2 + i)(A^i - 1) 



P{N^-l,N2 + VA) 



- [E21 + E12 + (El + E2)N2{N - N2)] P{Ni,N2;t) (43) 
If we use the relation iVi + A^2 = ^ (i.e.,iVi = N — N2), we can write: 
d 



dt 



P{N2;t) 



E21 + ^ {N2 - 1){N ~ N2 + 1) 



El 



E12 + -j^ {N2 + 1){N - N2 - 1) 



P{N2-l;t) 
P{N2 + l;t) 



E21 + E12 



(El + E2 

N 



■N2{N-N2) 



P{N2\t) (44) 



To obtain statements for the deterministic case, we define the mean value: 

oc 

{N2{t)) = J2 N2P{N2;t). (45) 

By multiplying the master equation with A'^2 and a following summation, we 
obtain: 



-{N2{t)) 



E2 ~^ E 



i {N2{N-N2)) + (^21-^12). 



Using the approximation {{N2Y) ~ (-^2)^ and the abbreviations 



X2 



(^2) 7P ni fl -^21 - E12 

; a = 1^2 - El ; fj ^ 



we achieve the corresponding deterministic equations: 

da;2 

— — = a a:;2 (1 - X2J + p 
dt 



(46) 



(47) 



(48) 



Let us compare the stationary behaviour of both the stochastic and the de- 
terministic model. In the case of two technologies we can explicitly derive the 
stationary solution of the master equation: 

P^(N2) ^ WHN2)WHN2-l)--WHN I) 

In the deterministic case the stationary solution for E2 > Ei is: 

a;2 = 1, that is N2 = N (50) 
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The new (better) technology will replace the old one. If a final stable station- 
ary solution is reached by the system all plants will use the new technology. 

For the stochastic case we get a quite different picture. In particular, the 
probability distribution P(A^2) 7^ for N2 ^ N . That means, in the stochastic 
case the old and new technology will coexist, given that the difference between 
El and E2 is not too large. Otherwise, the old technology will only survive 
in a small niche. 

An interesting case occurs, if the spontaneous rates are E21 = -B12 = 0. 
In this case the states TVi = 0, 7V2 = are called absorber states. This 
means, the system can not leave these states. For the stationary solution of 
the master equation we get: 

P°{N2) = CTi 5m2 + 0-2 5nm2 ;cri + 0-2 = 1 (51) 

Gi is a real number between zero and one, From the absorber states N2 = 
and A^i = {N2 — N) the other states can not be reached. 

From the initial state follows, which stationary solution is occupied by 
the system. After some calculations we can give for 02 in the limit TV 3> 1 
and N2{t = 0) the following equation: 

r for E2 < El 

0-2 = < /p \^2(o) (52) 

11 - f ^ 1 for Ko > Ri ^ ' 



E2 



with iV2(0) ~ N2(t ^ 0) the initial state of the system. Is ^^2(0) the number 
of users at time t = of the technology 2, then <T2 is the probability, that for 
t ^ 00 N2 ~ N users change to technology 2. ci is the probability that for 
t ^ 00 N2 = (i.e., A^i = N) users changes to technology 2, this means the 
new technology has not survived. In general, Ci is the survival probability of 
technology i. 

If a small number of plants A^2 (0) use a new technology, this technology 
will disappear if its growth rate E2 is smaller than that of the old one. If 
the growth rate E2 of the new technology is considerable larger, the new 
technology will be successful with the probability. 

Then, with the small probability: 

P N N2{0) 

S) '-' 

the old technology will still be used in the system. 

In the deterministic case a new technology with higher growth rates is 
always successful. This is the case of pure Darwinian selection where the 
fittest and only the fittest survives at the end of the process. 
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What we observe in socio-economic system is usually a variety of technolo- 
gies |l()lj . This can be either explained due to the action of mutation processes 
and error reproduction or due to the presence of stochastic processes. As we 
showed above in the stochastic situation the new better technology will only 
survive with a certain probability. Empirical studies of growth processes in 
ensembles of technologies might shed light on which growth mechanism is 
present in a certain system. 

3.3.2 Technologies with Quadratic Growth Rates In the following 
we consider again the case with absorber states, that means we neglect the 
terms E21, E12. As an extension to the model introduced above, instead 
of the linear terms {Ei, E2) we introduce now non-linear components (61, 
1)2^ V). V is a. parameter which in biochemical applications stays for the 
spatial volume of the system. In general terms it is related to the size of the 
system. Remember that we still have a system with a constant total number 
of elements (plants). So all processes appear as transition processes. The non- 
linearities might stay for processes of catalytic self-reproduction as well as for 
transition or mutation processes influences by other types (technologies) . We 
can write the transition probabilities as: 

W+{N{) = ^N2 N? (55) 

W~{N,) = ^ Nl m (56) 

(compare I36I in detail for an application to biochemical systems). We assume 
that the sum of the occupation numbers remains constant: 

Nl + N2 ^ N ^ const. (57) 

The probability distribution of the occupation numbers is: 

P{Ni,N2;t) (58) 

Analogous to the above chapter we can formulate the master equation as 
follows: 

^P{Ni;t)=W+{Ni-l)P{Ni-l;t) + W' {N^ + 1) P{Ni + l;t) 
at 

- [W+iNi) + W-iNi)] P{Ni-t) (59) 

By multiplying the master equation with Nk/V and summing over all oc- 
cupation numbers, we can get after factorization of the mean values the 
deterministic equation: 

TV 
{Nk)/V — Xk with xi -I- a;2 = — = C = const. 

Xi = hi xf — (fi Xi ; i = 1,2 (60) 
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with 

^ = M±M (61) 

Equations of the same form have been derived for so-called hypercyclic system 
to describe the evolution of macromolecules j40| . Their behaviour is very well 
understood. As a result of the quadratic terms in the growth rates the phase 
space is split into two regions separated by a separatrix Sf. 



Xi — U Oi Xj — O 

Fig. 12. Phase space in the deterministic case 



Let us also note here that because of the N equal constant condition the 
dimensionality of the system reduces. Due to the nature of the transition 
process, the occupation numbers in the system change along a diagonal of 
the two-dimensional phase space. In the deterministic picture, the selection 
behaviour depends on the initial conditions Xj (t = 0) of the system. A certain 
technology i only can win [xi = C for t — > oo) , if the initial condition places 
the system beyond the point of the separatrix: 

x,(0) > ^, ; S, = -^^ (62) 

O2 -r Oj 

This way it is possible that a new technology z, even when having a greater 
growth rate, will not win because at the beginning the number of users 
(plants) is too small. 

x,(i = 0) < S, (63) 

Such a situation has been called once-forever selection or hyperselection. In 
the case of macromolecular evolution this feature was used to explain the 
uniqueness of the genetic code. For technological evolution we have argued 
elsewhere that hyperselection is an alternative explanation for so-called lock- 
in phenomena of technologies 158l6j . The situation changes again if we look 
at the stochastic picture. Here the phase space looks as follows: 



iV2 = ^2^ N2 = N = CV 

Fig. 13. Phase space in the stochastic case 
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The two absorber states for the stochastic case are again: 

N2^0 , Ni = N , 
N2= N , Ni = . 
The initial state is N2it) = iV2(i = 0): 

P{N2;0) = <5jv,iv.(t=o) • 
The final state is: 

P{N2;t = oo) = (Ji 5qn2 + 0-2 ^NN2 ■ 



(64) 
(65) 

(66) 

(67) 



After some calculations we get for the case N2{Q) = 1 (one user of technology 
2 occurs at t = 0): 

0-2 = . . M-^ (68) 



02 

(T2 is the probability, that the new technology wins the competition process. 




f^ 



100 bi 



Fig. 14. Survival probability of a new technology in systems of different size (fat 
line N = 10, dashed line iV = 40, dotted line N = 70, fine line A^ = 100) 



In the stochastic case, the separatrix S is penetrated with a certain prob- 
ability. The once-forever behaviour disappears. The better technology can 
win the selection process with certain probability even if it starts only with 
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one user at the beginning. This probability increases rapidly for small overall 
numbers of users, as follows from the equation and as it can be seen in Fig. 1141 
\l^^ . In the stochastic description, the presence of fluctuations is responsible 
to help the technology to cross the barrier for entry in the market. 

This effect is size dependent. In smaller systems the probability for sur- 
vival of new technologies even in a hyperselection situation increases. This 
can be also interpreted in economic terms. A small size of the system refers 
to the size of the market in which the competition between the old and the 
new technology takes place. If a new technology is protected by certain mech- 
anisms it can win the process in a market of limited competition. This way it 
can obtain a significant number of early adopters and then go to a more open 
and bigger market with an improved chance of survival. In other words, in 
small niches, even in a lock-in situation, a new technology can replace the old 
one. By travelling from one niche to another or in growing niches (dynamic 
niche), the new technology would even be able to get control over the whole 
market. There is some empirical evidence that such processes take place in 
technological change (for a discussion see ^\)- 



3.3.3 Technologies with Mixed Growth Rates In the following, we 
consider the general case where the growth rates of a certain technology 
contain both linear as well as non-linear terms. The transition probabilities 
for this case are: 



W+{N,) ^^ N,N2 + -p^ N2 Nf (69) 

W-{N,) ^^N^N, + ^N, Nl (70) 

Again here we assume that E12 ~ E21 — 0, so we have absorber states. 
Further, the number of elements (plants) in the system is constant: N1+N2 — 
N — const.. In analogy to the case studies above we can write the master 
equation: 



^PiN,;t)^W+iN,-l)P{N,~l;t) + W' (N, + 1) PiN, + I't) 
ot 

- [W+{Ni) + W-{Ni)] PiNi;t) (71) 

In the same way as describes above we can derive the deterministic counter- 
part to the stochastic dynamics. 
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For the general case the survival probability for a new technology <Jn2(o).n 
can be calculated: 



Ar2(0)-1 3 

1+ 1. n 

j = l 4=1 


, N-i 
E2 +^277 


14 


N — i 

j=l ^=1 E2+b2 — 



^JV2(0),Af - /v _ ,■ — C^^) 



This result was first obtained in 36 and extensively applied to socio-economic 
problems in JH]. As can be seen from the formula above, the survival prob- 
ability for a new technology a not only depends on the selection advantage 
(the relation of the parameters Ei, hi), but also on the size of the system N 
(the overall user number) and the initial number of users. 

3.3.4 Summary General Aspects of the Survival Probability of a 
New Technology The stochastic analysis of competition processes of tech- 
nologies in a market where the number of overall users is restricted shows 
remarkable differences to the results known from a deterministic description. 
Let us note here that with the restriction to markets with a fixed number of 
economic agents, we restrict ourselves to technological substitution processes. 
However, using the approach of under-occupied systems this restriction does 
not represent a limitation for the innovation process. A variety of future pos- 
sible innovative types is included in the system all times. The population 
of agents (fixed in size) can unlimited travel over the field of innovations. 
Already with this restriction the stochastic analysis compared with the de- 
terministic one represents an increase in mathematical complexity at the 
descriptive level. In the multidimensional general case an analytical solution 
of the master equation is not available. Some results can be obtained using 
computer simulations |106l57ll'5] . However, what we showed so far is how 
statements about the survival probability in the long run can be derived. 
Results are available for certain special cases as we presented above. Let us 
summarize the results for the special case of two-dimensional systems with 
Ni + N2 = N — const, and with the two absorber states N2 = 0, N2 — N. 
Absorber states means here that once the states A'2 = or iV2 = A^ are 
reached, they cannot be left anymore. 

As mentioned above, this case can be treated exactly |l()fil,'ffil27] . Accord- 
ing to the absorber character we may assume that the stationary probability 
which is the target of evolution has a delta-character and can be written as: 

P{N2;t^oo) = a Snn2 + [1 - (t) Son^ ■ (73) 

Here, a is the survival probability of the second technology which is supposed 
to be the new one entering the market. An expression for the survival prob- 
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ability a can be calculated with the help of one constant of motion for the 
general case ^H] : 

JV2(0)-1 ] 

a^,(0),jv = '^\ ; ' for < N2{Q) < N (74) 

and 

'^w.(o),iv = 1 for A^2(0) = N . (75) 

As we introduced in subsection 13.2.11 the transition probabilities W^ and 
W~ represent quite a variety of different processes which all are linked to 
certain parameters. Further, the initial conditions and the system size N are 
important for the survival of a new technology (see for details 16 ). 

In particular we considered the following representations of transition 
probabilities: 

1. For technologies with linear growth rates (linear case) 

Wl = E2 -^ N, ;W^^ = E, -^ N, (76) 

2. For technologies with quadratic growth rates (quadratic case) 



NV 2 ; TVs 1 j^y 



^l = ^2 -^^ Ni ;Wi^^ = 6i ^ ,,,/' N2 (77) 



N 


~N2 




N 


N 


-N2 



3. For technologies with certain mixed growth rates (general case) 

Wl = E2 --^^ N2 + b2 ^^^ Ni (78) 

M/- = El '-^ N2 + bi ^ ^ N2 (79) 

We derived certain expressions for the survival probabilities by solving 
the absorber problem. These expressions can be treated further and certain 
special cases can be discusses. 

1. For the linear case the survival probability has been defined as 

/ 77. \ Af2(0) 
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We now consider the special case of very large systems (many possible 
users). In this case a can be simplified to: 

o-A'2(o),JV^oo = for E2 < El (81) 

and 



'^N,(0),N^oo = 1 - ( ;^ ) for E2 > El. (82) 

If we then consider a certain initial condition, the case that a technology 
starts with just one user (plant) N2(0) = 1, we can reduce the expression 
for the survival probability one step further 

crN2io)=i,N^oo = for E2 < El (83) 

El 
o-Ar2(0)=i.JV^oo = 1 - TT ioT E2 > El . (84) 

2. For the quadratic case the survival probability has been defined as 

(^N2{0),N - ; ^ N-l (^^) 

This equation can be written down more specified if we consider a new 
technology starting with just one user N2{0): 

<^N2{0),N = — , , jv_i (86) 

-^, 

3. In the general case the survival probability has the following form. 

N -i 



N2(a)-i 3 El +bi 

1+ T. n '- 



3=1 i=i E2 + 02- 



N-i 3 El +bi 



1+En 



V 



1 

3=1 i=l £'2 + 62-77 

In the special case of a technology starting with just one user 7V2(0) = 1 
we obtain: 

^JV2(o),Af = jY _^ (88) 

N-i 'J El +bi 



1+En 



V 



1 

3 = 1 i=l E2 + 62 



V 
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In general, the transition probabilities depend on the system size, the sys- 
tem parameters and the initial conditions. In the stochastic case we obviously 
find a niche effect. In the niche, the sharpness of selection is diminished. In 
the linear case, good and bad technologies can temporally coexist. In the 
quadratic case, the once-forever effect of a competition and selection process 
is countered by the niche effect. The niche represents a possibility for a better 
technology to win the competition even in the situation that a lock-in effect 
can be observed. Locally developed niches may play a constructive role in the 
technological evolution and can be observed in large complex systems, such 
as economic systems. The description, with the help of the master equation, 
includes processes, which play a role in these small domains (niches). In a 
niche, the new technology is protected against extinction for a limited time. 
After winning the competition in this small area, the new technology can 
infect the whole system, and may be established at the end. 

3.4 Stochastic Analysis of Multiple Decision Processes the 
Modelling of Technological Networks 

3.4.1 Ehrenfest's Urn-Models with Higher Correlations In subsec- 
tion l3.1.ll we introduced the urn-model of Ehrcnfcst.The idea of the Ehrenfest 
urn model was extended towards processes which also change the total num- 
ber of elements in the system. In the last subsections we restricted ourselves 
to the Ehrenfest approach by considering transition probabilities only, and 
derived analytical results for the survival probability for the special case of 
two competing technologies. In this section we use again the idea of a system 
with constant size, but extend the dimensionality of the system. Here, we 
have in mind under-occupied systems where the number of possible differ- 
ent technologies is large. Only the number of searching plants is restricted. 
Now we will consider the competition between s different technologies. This 
corresponds to the existence of s urns, which are filled with Ni, N2, ■ ■ ■ ,Ns 
spheres. We start a stochastic game where, at random times, spheres are 
taken out of an urn and put to another urn. Here we consider a game with 
binary decisions, where the transition from one urn j to another urn i is given 
by the following transition probabilities. 

W{Ni . . . , TV, -t- 1, . . . , TV, - 1, . . . , iV,|iVi, ...,N„...,Nj,...,N,) 

= A,, Nj + B,j N,Nj + J2 C'vk N,N,Nk (89) 

k 

This is a generalization of the transition rates introduced so far in the case 
of two technologies. In particular, we have in mind the mutual support or 
hindering of technologies. As discussed in the economic literature, innovations 
are often the outcome of an innovation network with different actors involved. 
We can assume that the decision of a certain plant or firm to implement a 
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certain technology depends both on other firms using the same technologies 
and also on other firms using different, but some how related, technologies. 
The stochastic game we propose here is a hopping process of plants between 
technologies. 

The mean values of the occupation numbers in the thermodynamic limit 
approximately follows a differential equation. In the sections above we showed 
such approximation for the case of two technologies with linear, quadratic and 
specific non-linear growth rates. The deterministic equations we obtained cor- 
respond to Fisher-Eigen-Schuster equations. For the generalization proposed 
above a set of Lotka-Volterra equation follows in the deterministic limit: 

Xi = > ' I Aij Xj + Bij NxiXj + y ' djk N^XiXjXk I (90) 



At 



/ I Aij Xj + Bij NXiXj -K y (-^ijk -'* XiXjXk I 
j \ k J 



with ' = Xi . 

The general case can only be handled by computer simulations. The equation 
above comprises the case of two technologies with non-linear growth rates if 
we introduce the following correspondence between parameters |31| : 

^21 = ; Bi2 ^ ^; Ci2k = 77 '^ife ; k = {1;2} (91) 

Ai2=0; B21 = ^■, C2ik - ^ 52k (92) 

W{N2 + l,N^-l\N^,N2) = E2 -^ + 62 -^ (93) 



3.4.2 Decision Processes and the Dynamics of a Network of Tech- 
nologies In this section we return to the general dynamics of interacting 
technologies. In subsection 13.2. II we gave a short economic interpretation of 
processes like spontaneous generation, self-reproduction, decay and conver- 
sion or transition. All these processes can be interpreted in terms of decision 
processes by firms or plants related to expansion or shrink and related to the 
choice of different technologies from a set of technologies available. The case 
of innovation interpreted as first occupation of a so far unoccupied technolog- 
ical possibility is just one process in a whole set of decision processes made 
by firms. The substitution case between old and new technologies that we 
discussed earlier in detail represents a very specific decision. In the following, 
we will give a detailed interpretation of different possible decision processes 
(for further economic interpretation please consult also |16p. 

We start again with the set of occupation numbers. Ni is the number 
of plants using a certain technology i. We consider a network of s different 



Modelling Selforganization and Innovation Processes in Networks 43 

technologies competing with each other i = 1, . . . , s. We will no longer stick 
to the assumption that the total number of plants in the system will remain 
constant. This way, we also consider growth and decline processes not only 
of certain used technologies in the system but also of the system, the market 
as a whole. 

We can differentiate between changes which take place inside one type of 
users (where the type or the group that the users belong to is characterized by 
the technology they use). For instance, the number of users of the technology 
i may increase or decrease. This is a stepwise process which only changes the 
occupation number by one: 

(N,) -^ {N, + l) 

(94) 
{Ni) -^ (N^-l) 

Furthermore, changes can take place between different types of users. 
Mathematically this is expressed by the simultaneous change of two occupa- 
tion numbers: 

'N,\ fN, - 1\ 

(95) 



NJ \N, + 1 

Plants can take decisions to leave a certain technology they have been us- 
ing before, they can develop a new technology, or use a technology already 
established in the market. They can also develop further a technology they 
have already used and in this way create a new type of technology. Not only 
technologies can enter the market; firms also can enter the market. The cre- 
ation of a new firm might be connected to a established technology, related 
to the introduction of a new technology. The problem technologies face by 
their introduction into a market are comparable to the problems firms have 
by entering a certain market. Here barriers for entry might also occur which 
hinder a certain technology in entering a market. On the other side, a network 
of firms might create support for a new firm, or also for a new technology, to 
enter the market (the system). Coalitions and cooperation are examples of 
synergetic effects in the introduction of new technologies. 

We will now formulate some possible changes in the language of transition 
probabilities: 

1. The number of plants using a certain technology i increases: 

iV, — > iV, + l (96) 

This change can be the result of different processes which are further 

differentiable. 
• A^Ni - linear self-reproduction: the number of plants will grow ac- 
cording to the existing number of plants. If these plants belong to one 
firm, firm growth is modelled. If they belong to different firms, the 
growth of an industrial sector is considered. The growth rate of this 
process is linear. Let us remember here that if only this process takes 
place in the system the technology would increase exponentially. 
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• AjNf - self-amplification (second order self-reproduction): here the 
number of plants already using the technology creates a network ef- 
fect of higher order which speeds up the growth momentum for this 
specific technology. 

• Bij NiNj - sponsoring or supporting from other plants: in this case 
plants using a different technology j are relevant for the growth of the 
technology i. One can think of systems of coupled technologies, one 
supporting the other, or of production chains where one technology 
relies on others. 

For this case the transition probabilities can be written: 

WiN, + l,N,\N„N,) = A'> N, + Al Nf + B,j N,N, (97) 

The different coefficients or parameters describe the strength of a certain 
effect which is acting in the system. 

2. Spontaneous formation of a new plant with a certain technology 

N, -^ N, + l (98) 

• <?o ~ spontaneous formation: here the entry of plants is not connected 
to the number of those already existing. One can think of a startup. 
The startup can either start with an already existing technology or 
be linked to the development of a new technology. In the latter case, 
Ni would be zero at the beginning. One would usually assume that 
the number of new plants created spontaneously is relatively low. 

W{N, + \,N,\N,,Nj) = <^o (99) 

3. Decrease of the number of the plants using technology i 

N, -^ N,-l (100) 

• D^Ni - linear decrease: Given that each technology in the market 
occupies a certain niche in the market, one can assume that the num- 
ber of plants using a technology which can not survive will depend 
on the size of the population of all plants using that technology. 

• DjNf ~ non-linear decrease or restricted capacity: this process re- 
mains for a network effect of the number of plants using a certain 
technology which drives plants out of the market. This process is re- 
sponsible for the existence of a restricted capacity in markets with 
linear growing technologies. Without the existence of decrease terms 
of higher order, one would be confronted with infinite, exploding mar- 
kets which stay in contradiction to empirical observations. 

W{N^-1,NJ\N^,NJ) = D° m + D} Nf (101) 
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4. Origin of a new technology connected to the formation of a new plant 
(induced innovation) 

• MijNj- Here we assume that the creation of a startup with a new 
technology is not a pure spontaneous process but is related to the 
number of plants using another (relevant for the new technology) 
technology. 

W{N, = 1,N,\N, = 0, N,) = My- N, (103) 

5. Change in the use of a technology (conversion, transition) 



N,\ fN^, 



No J \N, - 1 



(104) 



AijNj - simple transition from j to i: in this case the decision to take 
over a new technology is only influenced by the number of plants using 
a certain technology. One can interpret this process in the following 
way. If the number of plants using the same technology increases, then 
the competition between these plants also increases and plants might 
be motivated to look for another technology to increase their chances 
on the market. In the case where the technology is not yet occupied 
(not yet invented), the transition will also create an innovation for 
the system. Let us note here that beside spontaneous generation and 
the induced innovation, this process is important for the exploration 
of new areas in the technological space. 

BijNiNj - transition from j to i, in addition is promoted by i: in gen- 
eral one can assume that plants do not act in isolation. Conversely, 
the information flows between firms and plants about market condi- 
tions and technological change, are an important part of economic 
processes. Important in the process we discuss here is the decision to 
use a certain new technology j (here we use new in the sense that the 
technology is new for the plant) which is related to the number of 
firms already using this technology. We can further assume that the 
number of firms using a technology can be interpreted as a measure of 
attractiveness of this particular technology. In this case, the process 
represents one form in which imitation can be modelled. 
CijkNiNjNk - transition from j to i, that in addition is promoted 
(sponsored) by j and k: this process represents one possibility to 
introduce a network effect in the decision process of one firm to use 
a certain technology. One can imagine that the technologies j and k 
are related in sense of a production chain or that they complement 
each other. 



46 Hartmann-Sonntag et al. 

W{N, + 1,N,-1\N„N,) 

= A,j Nj + B,j N,Nj + Y^C.jk N,N,Nk (105) 

k 

The introduced model is composed in a modular way. Different processes 
related to the change in the occupation number space have been introduced. 
We also called these processes elsewhere elementary processes. In the presen- 
tation above we tried to give examples of processes relevant in the decision 
behaviour of firms using different technologies. However, alternative defini- 
tion and the introduction of further processes are possible within the model 
framework. The task consists of the definition of processes which can be 
observed empirically in the economy. The model represents a specific way to 
operationalize processes of decision making inside firms, the information flows 
between firms and the interactive pattern between technologies. By relating 
all these processes to the occupation number space a certain reduction of in- 
formation takes place. On the other hand, the different parameters allows for 
the possibility of including further economic information. The advantage of 
the modular structure of the model is that it puts different processes together 
and places them in an evolutionary framework. Growth of firms, substitution 
processes, invention and imitation, startups and firm closing are all part of 
one model. If we link the model to its deterministic counterpart, the instru- 
mentarium of dynamic systems becomes available. This way, at least, we can 
hope to get some insights in the analytic structure of the model and possible 
stationary states as well as their stability behaviour. Using the stochastic 
model for simulations we can obtain a lot of statements about the systems 
behaviour. Interesting investigations can be made by variation of the param- 
eters. Different kinds of connections can be analyzed. 

Beyond the economic interpretation chosen for the model, the framework 
can be also applied in quite different contexts. In any of these new application 
areas types, elements and the network of connections have to be interpreted at 
new again. Some of the authors applied the model for biochemical processes 
|H5| . for growth, competition and evolution of scientific specialties ^Hli and 
for the dynamics of values and competences |1()4| . 

3.4.3 Further Analysis of the ProbabiUty Distribution In the case 
that we obtain the probability distribution P{Ni, . . . ,Ns;t) analytically, or 
by computer simulations, it is possible to get the time-behaviour of the mo- 
ments (mean values, correlation functions) in dependence on the parameters 
of the system using the generation function |57I35| . For the partial differen- 
tial equation given in section 13.2.11 we can get an approximate solution by 
introducing the transformation 90 

F = exp[if(?7,)^] ; V^ = s^ - I (106) 
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at which the function ipirii) is expanded in a Taylor series 

<f == ^ a, ?7, + — ^ hj ri.Tjj + ^Yl ^'ok ViVjVk + •■■ (107) 

i i,j ij,k 

The coefficients a^ and bij are in terms of the moments of the probabihty 
distribution as follows: 

(108) 
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im] (109) 

(110) 

These equations give us information about the deviation from the Poisson- 
distribution (Poisson-distribution means for the coefficients bij =0, V i, j and 
all exponential coefficients of higher order are also zero). From the equations 
we can get a closed set of differential equations for the mean values Ui and the 
variances bij . Provided that we can neglect the coefficients higher than second 
order, we can achieve an approximative solution of the set of differential 
equations. Especially for processes which satisfy a multi-Poisson distribution, 
the coefficients higher than first order are zero in the Taylor expansion. This 
is the case if we get for F or 'F, respectively, a partial differential equation 
linear in Sj and at most of first order. If we know in this case the mean 
values Qi = {Ni)/N and start at t = with a multi-Poisson distribution, we 
can get an exact solution for the probability distribution. The solution is a 
multi-Poisson distribution for all times 

PiN;t) = n ^^^^ eM-im] (111) 

i 

Now we answer the question, which processes (transition probabilities from 

subsection 13. 2. l|l lead to a multi-Poisson distribution, i.e., which processes 

have in the generating function at most terms with a first derivative with 

respect to Si and are linear in s^. Then, the generating function looks as 

follows: 

,_, ar^ ,„, fiTP 

(112) 
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If we start at t = with a niulti-Poisson distribution, the probabihty distri- 
bution remains a muhi-Poisson distribution for all times 

PiN:t) = cxp[-(7V(i))c] n ^^'^r' (115) 

i 

The extinction probability of the whole component is 

P(0;t) = exph(iVc(i))] (116) 

This extinction probability depends on the overall number of individuals Nc 
of the component and decreases exponentially with {Nc{t)). 

A detailed description for the calculation of the moments of the probabil- 
ity distribution P(Ni, . . . , Ns;t) and the probability distribution was given in 
the work of Heinrich and Sonntag 1.57^ . The time-behaviour of the moments 
could be achieved with the help of generating function. Deviations from the 
Poisson behaviour were investigated. Fluctuations and their influence on the 
system structure and behaviour could be analyzed. The analysis of time- 
dependent correlation functions could also be interesting for the description 
of socio-economic systems. 

4 Summary 

This paper is devoted to the study of innovation processes in socio-economic 
contexts. In particular, we investigate the influence of stochastic effects on 
processes of self-organization and evolution. We take a special network per- 
spective. Starting with the new emergent field of complex networks theory, 
we develop our own approach of sensitive networks relevant to the description 
of an innovation. 

Innovation is first introduced on a general level as a specific process that 
changes the composition and dynamic constitution of a system. We use a 
discrete representation of the system in terms of a space of occupation num- 
bers. Then, innovation can be described as a hopping process between positive 
cones. Further, we introduce the notion of an under-occupied system. This 
way we implement a set of possible future paths of developments in our mod- 
elling. We relate this abstract notion of an innovation to the discussion of 
innovation processes in economics. 

In the economic literature, innovation has been understood as the outcome 
of processes running on networks of different actors. We concentrate on firms 
and technologies in this paper. We present a network theory of innovation by 
mapping the dynamic interactions related to the emergence of an innovation 
in form of a graph. 

Figure 1151 illustrates our approach. The system is composed of a large 
set of enumerable types. Each of these types is represented by a node. At 
a certain point in time, only a small part of these nodes are active. The 
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pattern of interaction between them (including processes of self-influence) 
determines the dynamic composition of the system. It is visualized in terms of 
(active) links between the nodes. This active network will produce a dynamics 
which has a certain set of stable states. We assume that the activated part 
of the network is embedded in a much larger network of inactive nodes and 
links. The inactive nodes represent future possibilities in the evolution of the 
system. An innovation appears when a unoccupied node becomes occupied 
for the first time. With this first occupation, the set of links connecting the 
"new" node with already occupied nodes also becomes activated. It is readily 
apparent that such an event changes the whole composition of the system. 
Accordingly, the stable state that the system might have reached already 
becomes instable and the system searches for a new stable state. If we assume 
that the interaction between the nodes (types) is a competitive one, the 
stable state of a certain activated network can also include the deactivation 
of certain nodes. Types (nodes) which are selected out will transit to a non- 
occupied inactive status. 

We first discuss structural (static) properties of such a relational network. 
In this way, instruments from random graph theory and percolation theory 
become relevant. In particular, formulae for the probabilities of the occurrence 
and the distribution of components and cycles in large networks have been 
obtained by combinatorial considerations. We show that connectivity is a 
central measure in the structure of these kinds of networks. Evolving, dynamic 
networks show different phenomena compared to random graphs. The results 
better correspond to findings in real data of empirical networks such as the 
appearance of power law distributions of the degree function. 

In the centre of the paper we turn to the discussion towards the dy- 
namic properties of networks. In particular, we derive descriptions for a net- 
work of interacting technologies and interacting firms. We call these networks 
sensitive because innovation processes described in terms of the removal or 
appearance of a node might change the dynamic behaviour of the system dra- 
matically. We use a stochastic description of such an evolving network and 
base this on the theory of birth and death processes. We introduce different 
forms of transition probabilities for closed systems as well as for open, growing 
and declining systems. We define master equations and non-linear differen- 
tial equations as their deterministic equivalents. Contrary to deterministic 
models, the stochastic description offers the advantage that, at finite times, 
new technologies (types) can arise or "die out" . Further, the emergence of an 
innovation can be treated as a singular stepwise event implying the transition 
from an under-occupied to an occupied state. 

In some special cases the master equation can be solved analytically and 
a stationary survival probability for an innovation can be derived. We show 
that the stochastic dynamics differ essentially from the deterministic. Sep- 
aratrices, which decompose the phase space, cannot be intersected in the 
deterministic case. In the stochastic case they can be crossed. This way, a 
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Fig. 15. Illustration of a sensitive network. Embedded in a large network of inactive 
nodes and links the activated part of the network (upper part of the figure) changes 
its composition when an innovation is emerging (lower part of the figure). 



"once-for-ever" or hyperselection that is known in economics as lock-in phe- 
nomena for technologies can be avoided. 

The stochastic model of netvirorked dynamic interactions of technologies 
is further generaUzed in a multi-dimensional case. Processes representing dif- 
ferent non-linearities are discussed in the context of technological change. 
Although we use technological evolution as main reference point, we also 
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point to the fact that the modular structure of the model also allows for its 
application in quite different fields. Applications in fields as biology, popula- 
tion theory and science of science have been presented |45I18| . 
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